Skip to content

Conversation

@bhuang95
Copy link
Contributor

@bhuang95 bhuang95 commented Dec 16, 2025

This PR works along with the following three dependent PRs to enable the EnKF-only configuration for the atmosphere within the global workflow (see detailed description NOAA-EMC/global-workflow#4345)

Dependencies:
-NOAA-EMC/global-workflow#4345
-NOAA-EMC/jcb-gdas#219
-NOAA-EMC/jcb#32

Resolve

CoryMartin-NOAA added a commit to NOAA-EMC/jcb that referenced this pull request Jan 8, 2026
This PR works along with the following three dependent PRs to enable the
EnKF-only configuration for the atmosphere within the global workflow
(see detailed description
NOAA-EMC/global-workflow#4345)

Dependencies:
-NOAA-EMC/global-workflow#4345
-NOAA-EMC/GDASApp#2010
-NOAA-EMC/jcb-gdas#219

Resolve
- NOAA-EMC/global-workflow#4339

---------

Co-authored-by: Cory Martin <cory.r.martin@noaa.gov>
@bhuang95
Copy link
Contributor Author

bhuang95 commented Jan 9, 2026

@CoryMartin-NOAA I updated this branch with develop. It now has issues of reading the obs distribution and localization blocks in atm_ens_obs_dist_localizations.yaml.j2
to configure LETKF yaml wtih below errors. This yaml is used on Line 23 of this LETKF YAML template. Do you know what recent changes may cause this? Looks like something is wrong with the format in this yaml?

1251 ^[[38;5;39m2026-01-09 22:53:31,307 - DEBUG    - jedi        : Writing JEDI YAML config to: /scratch4/BMC/gsienkf/Bo.Huang/expCodes/Workflow/EnKFOnly-20251031/TestGWSet     upRealCase/tmp/RUNDIRS/TestGW-EnKFOnly-JEDI-MEM3-T11/enkfgdas.2022010318/enkfgdasatmensanl_18/atmensanlobs.yaml^[[0m
1252 Traceback (most recent call last):
1253   File "/scratch4/BMC/gsienkf/Bo.Huang/expCodes/Workflow/EnKFOnly-20251031/global-workflow/scripts/exglobal_atmens_analysis_initialize.py", line 26, in <module>
1254     AtmEnsAnl.initialize()
1255   File "/scratch4/BMC/gsienkf/Bo.Huang/expCodes/Workflow/EnKFOnly-20251031/global-workflow/sorc/wxflow/src/wxflow/logger.py", line 252, in wrapper
1256     retval = func(*args, **kwargs)
1257              ^^^^^^^^^^^^^^^^^^^^^
1258   File "/scratch4/BMC/gsienkf/Bo.Huang/expCodes/Workflow/EnKFOnly-20251031/global-workflow/ush/python/pygfs/task/atmens_analysis.py", line 90, in initialize
1259     self.jedi_dict['atmensanlobs'].initialize(clean_empty_obsspaces=True)
1260   File "/scratch4/BMC/gsienkf/Bo.Huang/expCodes/Workflow/EnKFOnly-20251031/global-workflow/sorc/wxflow/src/wxflow/logger.py", line 252, in wrapper
1261     retval = func(*args, **kwargs)
1262              ^^^^^^^^^^^^^^^^^^^^^
1263   File "/scratch4/BMC/gsienkf/Bo.Huang/expCodes/Workflow/EnKFOnly-20251031/global-workflow/ush/python/pygfs/jedi/jedi.py", line 189, in initialize
1264     save_as_yaml(self.exe_config, self.jedi_config.exe_config_yaml)
1265   File "/scratch4/BMC/gsienkf/Bo.Huang/expCodes/Workflow/EnKFOnly-20251031/global-workflow/sorc/wxflow/src/wxflow/yaml_file.py", line 54, in save_as_yaml
1266     yaml.safe_dump(vanilla_yaml(data), fh,
...

1320   File "/contrib/spack-stack/spack-stack-1.9.2/envs/ue-oneapi-2024.2.1/install/oneapi/2024.2.1/py-pyyaml-6.0.2-55i3htn/lib/python3.11/site-packages/yaml/representer.py     ", line 58, in represent_data
1321     node = self.yaml_representers[None](self, data)
1322            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1323   File "/contrib/spack-stack/spack-stack-1.9.2/envs/ue-oneapi-2024.2.1/install/oneapi/2024.2.1/py-pyyaml-6.0.2-55i3htn/lib/python3.11/site-packages/yaml/representer.py     ", line 231, in represent_undefined
1324     raise RepresenterError("cannot represent an object", data)
1325 yaml.representer.RepresenterError: ('cannot represent an object', {'name': 'RoundRobin', 'halo size': '1250e3'})

@CoryMartin-NOAA
Copy link
Contributor

@bhuang95 I ran into this same problem on Thursday. I don't immediately know what new change broke this, but I think the fix is this: NOAA-EMC/wxflow#65

DavidNew-NOAA pushed a commit to NOAA-EMC/jcb-gdas that referenced this pull request Jan 16, 2026
This PR works along with the following three dependent PRs to enable the
EnKF-only configuration for the atmosphere within the global workflow
(see detailed description
NOAA-EMC/global-workflow#4345)

Dependencies:
-NOAA-EMC/global-workflow#4345
-NOAA-EMC/GDASApp#2010
-NOAA-EMC/jcb#32

Resolve
- NOAA-EMC/global-workflow#4339

---------

Co-authored-by: Cory Martin <cory.r.martin@noaa.gov>
DavidNew-NOAA pushed a commit that referenced this pull request Jan 16, 2026
This PR works along with the following three dependent PRs to enable the
EnKF-only configuration for the atmosphere within the global workflow
(see detailed description
NOAA-EMC/global-workflow#4345)

Dependencies:
-NOAA-EMC/global-workflow#4345
-#2010
-NOAA-EMC/jcb#32

Resolve
- NOAA-EMC/global-workflow#4339

---------

Co-authored-by: Cory Martin <cory.r.martin@noaa.gov>
@CoryMartin-NOAA
Copy link
Contributor

@bhuang95 does any change here depend on global-workflow or is it the other way around? Meaning, can we (after you or I resolve conflicts) work towards getting this merged in ASAP?

@bhuang95
Copy link
Contributor Author

@bhuang95 does any change here depend on global-workflow or is it the other way around? Meaning, can we (after you or I resolve conflicts) work towards getting this merged in ASAP?

@CoryMartin-NOAA These two lines with default value: false below should remove the dependency of this PR on the global workflow, and the other way around.

{% if DOENKFONLY_ATM | default(false) %}

atm_enkfonly: {{ DOENKFONLY_ATM | default(false, true) }}

The other major changes about obs distribution and localization will be handled in the NOAA-EMC/jcb#32 (merged) and use atmosphere (without obs localization setting) directory instead of atmosphere-lgetkf https://github.com/NOAA-EMC/GDASApp/tree/develop/parm/jcb-gdas/observations. I think both will not affect the upstream and downstream either.

I addressed the conflicts. I think it should be ready to merge if it looks good to you.

Copy link
Contributor

@CoryMartin-NOAA CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks @bhuang95 I'm going to test this and then approve if it works well

# Observation things
# ------------------
{% include OBS_LIST_YAML %}
{% include OBS_DIST_LOCALIZATIONS_YAML %}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does seem that this requires global-workflow to be updated/synchronized, because otherwise this variable is not defined.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CoryMartin-NOAA That's a bummer. Thinking about two options,
Option 1:

  • Remove Line 20 and add a TODO comment below Line 14
  • Change Line 3 to app_path_observations: {{PARMgfs}}/gdas/jcb-gdas/observations/atmosphere-lgetkf

Option 2:

  • Add this line below in job-base.yaml.j2 and replace OBS_DIST_LOCALIZATIONS_YAML with "obs_dist_localizations"
  • obs_dist_localizations: {{ OBS_DIST_LOCALIZATIONS_YAML | default("{{PARMgfs}}/gdas/atm/atm_ens_obs_dist_localizations.yaml.j2", true) }}

Would any of these work? I am also fine to wait for global-workflow.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like Option 2 if it works, because that means we can get rid of the observations/atmosphere-lgetkf directory sooner rather than later

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think they will be more static than the list of obs

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disregard my deleted comments. I concur with Cory that these parameter should be in jcb-base.yaml.j2

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CoryMartin-NOAA @DavidNew-NOAA Looks like we can simply add {% include 'atm_ens_obs_dist_localizations.yaml.j2' %} in the LGETKF yaml. This change passed my JEDI LGETKF run on Ursa.

Copy link
Contributor

@CoryMartin-NOAA CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved but we need to not merge until the workflow PR is ready or else things will be broken

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants